The present invention relates generally to image processing, and more specifically to high speed image enhancement by dynamic range adaptation and normalization and high fidelity color reconstruction.
The three major factors that define image quality are: 1) dynamic range (DR), which is the number of distinct intensity levels in each spectral band that is needed in order to carry all the image data; 2) color fidelity, which is how close the reconstructed image colors are to the true life scenery colors; and 3) resolution, which is the smallest observable (to a viewer) detail in the image.
The DR of a display, e.g. of paper sheets and electronic screens, is the number of intensity levels perceived by the eye on the display. Due to a well-known psychophysical limitation, the DR of common displays is smaller than the DR of most photographed images. As a result, many details are not accessible to our eyes and simply get lost when a photograph is displayed on a screen or printed on paper. Thus, dynamic range compression or adaptation is needed in order for the viewer to be able to actually see all the data that is in the image.
The DR in natural scenarios can exceed 1010. The DR of the neural channel from the human eye to the brain is physically constrained to less than 1000. Thus, it has been conjectured by researchers that dynamic range compression (DRC), which is using less intensity levels to carry more data, must take place somewhere between the eye photoreceptors and the neural communication channel input.
Continuous research on the human, as well as on animal visual systems has been conducted since the mid 19th century. Numerous models have been suggested to explain the outstanding ability of biological eyes to adapt to tremendous dynamic ranges of luminance. Many researchers agree that the DR adaptation of biological eyes is performed in a neural network located just beneath the photoreceptors in the retina. (Dowling, J. E. et al., “The interplexiform cell: a new type of retinal neuron,” Invest. Opthalmol. Vol. 15, pp. 916-926, 1976. )
The two most remarkable phenomena associated with the retina are: 1) the apparent unlimited range of DRC ratios that the eye is capable of exploiting, which is modeled by the Michaelis Eqn., also known as the Weber's Law (Hemila S., “Background adaptation in the rods of the frog's retina,” J. Physiol., Vol. 265, pp. 721-741) and 2), the relation between the spatial acuity, i.e., the effective spatial resolution of the eye, and the average luminance level of the scenery.
It has been discovered (Van Nes, F. L. et al., “Spatial modulation transfer in the human eye,” J. Opt. Soc. Am., Vol. 57, pp. 401-406, 1967; and Westheimer, G., “Visual acuity and spatial modulation thresholds,” Chap. 7 in Handbook of Sensory Phys., Vol. VII/4, pp. 170-187, Springer Verlag, 1972) that the effective resolution of the eye is monotonically increasing as a part of the adaptation to increasing levels of luminance.
A spatial feedback automatic gain control (AGC) model has been suggested and investigated by Shefer (Shefer M., “AGC Models for Retinal Signal Processing”, M.Sc.Thesis, The Technion—Israel Institute of Technology, November 1979.)
This model simultaneously explains several important properties of the spatio-temporal response of the eye. In particular, it explains how both Weber's Law and the spatial frequency response shift with luminance, are generated as a part of a simple unified model.
The AGC model realization for DRC has been suggested in patents by Shefer (Israel Patent No. 109824, U.S. Pat. No. 5,850,357) and by Zeevi et al. (Canadian Patent No.1.318.971). Zeevi suggests an implicit recursive realization that takes time to converge to a sufficiently accurate solution. This recursive algorithm is clearly inferior to closed form algorithms that compute the desired result explicitly and non-recursively based on the known input only.
In his patents, Shefer suggested a closed-form solution for the AGC model. However, such a closed-form solution is only attainable for a uni-dimensional case. Hence, applying it to image processing implies that it should process the rows and the columns of the image separately and sequentially. This again is a time and memory consuming method that does not fit either real-time processing of video, or processing of large batches of images.
In U.S. Pat. No. 5,991,456, Z. Rahman et al. propose a Homomorphic configuration based filter for combined improvement of DRC, color constancy and lightness rendition aspects. This combined treatment of all three aspects together creates a three-way coupling that makes it difficult to optimize each aspect separately. Hence, the final result must be some compromise between the three aspects. Also, the Rahman filter does not exhibit a Weber Law behavior, nor does it have the adaptive resolution property that comes along with it, unlike AGC-based solutions. As a result, the Rahman filter response, although logarithmic in nature, presents a rather limited range of DRC ratios along with a rather constant resolution, which combination does not resemble the human eye operation. Hence, the result cannot look as natural as it otherwise would.
Other disadvantages of the Rahman filter are the fact that each spectral band has to repeatedly undergo the whole filtering process quite a large number of times, each time with a different size of the convolution neighborhood. This results in a very computational-intensive and time consuming routine, that makes the color reconstruction process cumbersome and difficult to optimize. Rahman's exponentially decaying neighborhoods also require added computation time and memory. This logic makes the Rahman filter unfit for real-time processing of video and fast processing of large batches of images.
In general, all DRC algorithms in the prior art that exhibit either a logarithmic or a Weber law type of response behavior, suffer from a common disadvantage: the DRC is only applied to the light areas of the image, resulting in more data being revealed in the darker areas of the image. Consequently, the visibility improvement in lighter areas may be insufficient.
There is thus a widely recognized need for, and it would be highly advantageous to have, an automated method of improving digital color images at high speed and for supporting pipe-lining, a method that has very little memory requirements.